Latent class regression model in IRLS approach

Keywords--Regress ion, Latent classes, Iteratively reweighted least squares. I. I N T R O D U C T I O N We consider simultaneous constructing of several regressions by subsets of a given data set. Such an approach corresponds to so-called latent class models known in various statistical applications [1-3]. Latent class techniques are applied in factor and scaling analyses [4-8], structural equations and latent s tructure analysis [9-11], and social modeling [12-14], where the te rm latent has a meaning of an unobserved variable. The concept of latent class is widely used in the marketing research field where the t e rm latent denotes the segments or subsets of da ta [15-20]. In this paper, we consider latent class regression modeling that has been studied in various works [21-26]. We suggest a new formulation of the maximum likelihood (ML) objective of the probabili ty that each observation belongs to at least one class. This formulation produces a solution expressed as an iteratively reweighted least squares (IRLS) procedure for solving nonlinear statistical problems [27-33]. We apply this technique to latent class regression models and to the estimation of the parameters of mixed distributions. In addition to regressions, we obtain the probabilities of belonging to each class for each observation. The paper is arranged as follows. Section 2 describes the regular max imum likelihood for linear regression and for parameters of normal distribution. Section 3 introduces max imum likelihood for probabil i ty tha t each observation belongs to at least one of several possible ciasses. Section 4 presents numerical examples for latent class modeling, and Section 5 summarizes. The authors wish to thank two reviewers for their suggestions tha t improved the paper. 0895-7177/05/$ see front matter @ 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.mcm.2005.01.031 Typeset by A3AS-TEX 302 S. LIPOVETSKY AND W . M. CONKLIN 2. M A X I M U M LIKELIHOOD FOR LINEAR R E G R E S S I O N Let us at first describe a regular maximum likelihood approach known in regression analysis for the linear model yi = ao + a l x i l + . . . --k anZin + ci, (1) where observations (i = 1 , 2 , . . . , N ) by the dependent variable yi are fitted with the linear combination of the observed predictor variables x i l , x i g , . . . , xin, (n -number of the independent variables), and ei denotes deviations from the linear form. Suppose the deviations si correspond to a random noise defined by a normal distribution with the probability density function (pdf) f@i) -x/27cr exp \ 2cr2j , (2) where cr is the standard deviation. Probability of the random event corresponded to an ith observation is As V/~C r exp \ 2o.2.,] , (3) where Ae denotes a constant infinitesimal interval around the random term. Likelihood of the event tha t all independent observations occurred is the product of all probabilities (3) that is the objective of maximum likelihood M L = ~ p , = r I ~ e x p \ 2cT2 j exp ~ 2 i=1 i=1 i=1 (4) Logarithm of the ML objective equals N lnML = N l n N l n a ~ zi --* max. (5)